Active Learning for Effectively Fine-Tuning Transfer Learning to Downstream Task

Language model (LM) has become a common method of transfer learning in Natural Processing (NLP) tasks when working with small labeled datasets. An LM is pretrained using an easily available large unlabelled text corpus and fine-tuned the labelled data to apply target (i.e., downstream) task. As designed capture linguistic aspects semantics, it can be biased features. We argue that exposing during fine-tuning instances diverse semantic (e.g., topical, linguistic, relations) present dataset will improve its performance on underlying propose Mixed Aspect Sampling (MAS) framework sample different use ensemble classifier classification performance. Experimental results show MAS performs better than random sampling as well state-of-the-art active models abuse detection where hard collect for building accurate classifier.